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1.  SUMMARY  OF  ADDRESSED  TASKS  AND  ACCOMPLISHMENTS 

We  have  directed  our  efforts  towards  solving  the  problems  associated  with  Objective  1 :  Devel¬ 
opment  of  General  Multidecision  Theory  of  Sequential  Hypothesis  Testing  and  specifically  Task 
1:  Development  of  Nearly  Optimal  Multidecision  Sequential  Rules  for  Testing  Multiple  Com¬ 
posite  Hypotheses.  In  particular,  we  proposed  three  sequential  tests  of  composite  hypotheses  and 
formulated  general  conditions  under  which  these  tests  are  asymptotically  optimal  when  error  prob¬ 
abilities  approach  zero.  Theses  conditions  are  related  to  the  strong  law  of  large  numbers  (SLLN) 
for  the  log-likelihood  ratios  of  hypotheses  and  rates  of  convergence  in  the  SLLN.  The  general 
theory  covers  a  variety  of  popular  models  such  as  Markov  and  hidden  Markov  models  and  their 
generalizations. 

2.  MAIN  RESULTS 

2.1.  General  Theory  of  Sequential  Testing  Multiple  Composite  Hypotheses 

2.1.1.  The  Problem  and  Structure  of  Sequential  Tests 

Consider  the  following  general  scenario  of  testing  multiple  composite  hypotheses  associated  with 
non-iid  stochastic  models.  Let  (fi,  ,  TPn,  P0),  n  =  1,2...,  be  a  filtered  probability  space 
with  standard  assumptions  about  monotonicity  of  the  cr-algebras  ■'Xn.  The  vector  parameter  G  = 
(0i, ... ,  Of)  belongs  to  a  subset  0  of  (-dimensional  Euclidean  space.  The  sub-a-algebra  = 
.'T-f  =  <x(X")  of  &  is  generated  by  the  stochastic  process  X”  =  (X1? . . .  ,Xn)n^i  observed  up 
to  time  n.  The  hypotheses  to  be  tested  are  “77,;  :  G  e  Of,  i  —  0, 1, . . . ,  N  (N  ^  1),  where  0.; 
are  disjoint  subsets  of  0.  We  will  also  suppose  that  there  is  an  indifference  zone  Im  e  0  in  which 
there  are  no  constraints  on  the  probabilities  of  errors  imposed.  The  indifference  zone,  where  any 
decision  is  acceptable,  is  usually  introduced  keeping  in  mind  that  the  correct  action  is  not  critical 
and  often  not  even  possible  when  the  hypotheses  are  too  close,  which  is  perhaps  the  case  in  most, 
if  not  all,  practical  applications.  However,  in  principle  Iin  may  be  an  empty  set.  The  probability 
measures  Pe  and  are  assumed  to  be  locally  mutually  absolutely  continuous.  By  pe(Xn |X”-1), 
nf  1  we  denote  corresponding  conditional  densities  which  may  depend  on  n. 

A  multihypothesis  sequential  test  5  =  (T,  d)  consists  of  the  pair  ( T,d ),  where  T  is  a  stop¬ 
ping  time  with  respect  to  the  filtration  and  d  =  dT(Xf)  e  {0, 1, . . . ,  Nj  is  an  .im¬ 

measurable  (terminal)  decision  rule  specifying  which  hypothesis  is  to  be  accepted  once  observa¬ 
tions  have  stopped  (the  hypothesis  Hi  is  accepted  if  d  —  i  and  rejected  if  dfi,  i.e.,  {d  =  i}  = 
{T  <  oo,  <5  accepts  Hi}).  The  quality  of  a  sequential  test  is  judged  on  the  basis  of  its  error  prob¬ 
abilities  and  expected  sample  sizes  or  more  generally  on  the  moments  of  the  sample  size.  Let 
G )  =  P 0(d  =  j)H{6»ee,}  (*  f1  j,  hj  =  0, 1, . . . ,  N)  be  the  probability  of  accepting  the  hy¬ 
pothesis  Hj  by  the  test  5  when  the  true  value  of  the  parameter  G  is  fixed  and  belongs  to  the  subset 
Oi  and  let  fdfd,  9)  =  P  e(d  f  i)l{6»e0i}  be  the  probability  of  rejecting  the  hypotheses  H,L  when  it  is 
true.  Introduce  the  following  two  classes  of  tests 

c(ll«ijll)  =  {<*  :  sup  0Hj(6,G)  f  aij,  i,j  =  0, 1, . . .  ,N,  i  f  j  }  , 

l  0S0;  J  ^ 

C(/3)  =  (<5  :  sup  /3i(S,G )  <  ff,  i  =  0,1,.. .  ,n\ 

l  0S0;  J 
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for  which  maximal  error  probabilities  do  not  exceed  the  given  numbers  and  33 1. 

The  Generalized  Matrix  Sequential  Likelihood  Ratio  Test  Define  the  generalized  LR  statistics 


suPee©  Uk=iPe(Xk\ 

x?-1)  nLiP^(A*i 

|x*- 

«uPflG0,  Urk=iP0(Xk 

IX1  *)  sup0ee.  Ylk=iPo(xk\ 

IX?-1) 

i  =  0, 1, . . . ,  iV,  (2) 


where  0*  =  argsup0g0p£»(X")  is  the  MLE  estimator.  The  first  multihypothesis  test,  which  we 
will  refer  to  as  the  Multihypothesis  Generalized  Sequential  Likelihood  Ratio  Test  (MGSLRT),  is  of 
the  form 

stop  at  the  first  n  ^  1  such  that  for  some  i  A3n  ^  A,J%  for  all  j  ^  i  (3) 

and  accept  the  (unique)  Hi  that  satisfies  these  inequalities,  where  Aij  are  positive  and  finite  num¬ 
bers  (thresholds). 

The  Adaptive  Matrix  Sequential  Likelihood  Ratio  Test  Let  0,,  =  0,,  {X\, . . . ,  Xn)  be  an  esti¬ 
mator  of  0  (not  necessarily  the  MLE).  If  in  conditional  density  p0(AA|X?_1)  for  the  kth  observation 
given  the  previous  data  X?-1  =  (A", , . . . ,  Xk_x)  we  replace  the  parameter  by  the  estimate  0k  _  | 
built  upon  the  sample  X?-1  that  includes  k  —  1  observations,  then  pgk  ^X^X?-1)  is  still  a  vi¬ 
able  probability  density,  in  contrast  to  the  case  of  the  GLR  approach  where  p~e  (Xfc|X?_1)  is  not  a 
probability  density  anymore  for  k  A  n.  Therefore,  the  statistic 


A  n(0i)  =  n 

k= 1 


p^.t^ix?"1) 


A„-i(6i)  x 


Po.-.i-v.ixr1) 

Pe,(x„|xr‘) 


(4) 


is  a  viable  likelihood  ratio,  and  it  is  the  nonnegative  ?oi -martingale  with  unit  expectation,  since 
E0;  [An^^lX™-1]  =  A*.,  (6f ).  Therefore,  one  can  use  Wald’s  likelihood  ratio  identity  for  finding 
bounds  on  error  probabilities  if  A  *  (0j)  is  used  instead  of  the  LR  with  the  true  parameter  value  0. 
Because  of  exactly  this  very  convenient  property  as  well  as  of  the  simple  recursive  structure  (4) 
the  hypothesis  tests  based  on  the  adaptive  LRs  with  one-stage  delayed  estimators  represent  a  very 
attractive  alternative  to  the  GLR  tests  as  well  to  the  mixture-based  tests  introduced  below. 

Define  the  statistics 


k— 1\ 


A*  = 


nLi  Pek_Mk\n 

k— 1 1 


sup0e0!  Wk=iPo{xk\^i 


i  =  0, 1, 


,N. 


(5) 


The  second  multihypothesis  test,  which  we  will  refer  to  as  the  Multihypothesis  Adaptive  Sequential 
Likelihood  Ratio  Test  (MASLRT),  has  the  form 


stop  at  the  first  a  A  1  such  that  for  some  i  AJn  A  Ajt  for  all  j  ^  i 


(6) 


and  accept  the  (unique)  Hi  that  satisfies  these  inequalities. 

It  is  convenient  to  re-write  the  MASLRT  (6)  in  the  following  form.  Introducing  the  statistics 


C  =  X1oSP4-.(aAIx“). 

k= 1 


n 

4  =  SUP  E  logMX.lX.t1) 

0e0i  fc=i 
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the  MASLRT  can  be  written  as 


where 


rj~\ * 


min  T* 

o^n  1 


1 


d*  =  i  if  T*  =  T*, 


(7) 


T*  =  inf 


n  A  1  :  t  > 


max  W  +  a,-*, 

OCjCN 


aij  =  logAjj,  i  =  0, 1, . . .  ,N. 


(8) 


The  Weighted  (Mixture-based)  Sequential  Likelihood  Ratio  Test.  Yet  another  approach  is  to 
use  mixtures  of  LRs  (weighted  LRs)  in  test  constructions.  This  approach  was  proposed  by  Wald  [6] 
in  his  seminal  work  on  the  SPRT  and  its  extensions  to  two  composite  hypotheses. 

Define  the  weighted  LRs 


A 


i 

n 


/ontiMAAixnfLw  .  = 

4nLi^(^ixn^(d0)’  : 


(9) 


where  the  weight  functions  W ( 0 ),  Wfd),  i  =  0, 1, . . . ,  N,  are  not  necessarily  normalized  to  1.  If 
the  weights  are  normalized  to  1,  then  they  can  be  regarded  as  probability  distributions.  Let  {A^} 
(■ i  f  j)  be  positive  numbers.  The  multihypothesis  weighted  SLRT  (MWSLRT)  5  =  (T,  d)  is  of  the 
form 

stop  at  the  first  n  A  1  such  that  for  some  i  ^  A]t  for  all  j  ^  /’  (10) 

and  accept  the  HL  that  satisfies  these  inequalities. 

Taking  logarithms  and  writing 


4  =  log  /  UMxk\xr)wm 

k= i 


=  log 


/  i[po(Xk 

&i  k=  1 


xf-^w^d^), 


the  MWSPRT  can  be  also  expressed  as 


where 


T  =  min  Tt  d  —  i  if  T  =  7) 

O^i^N 


Tj  =  inf  <  n  >  1 


A  max  [E  +  log  An] 
i¥=i 


i  —  0, 1, , . . ,  N. 


(ID 


(12) 


2.1.2.  Probabilities  of  Errors 


One  of  the  most  important  issues  is  to  obtain  upper  bounds  and  approximations  for  error  probabil¬ 
ities  of  the  introduced  tests.  However,  we  do  not  know  how  to  upper-bound  the  error  probabilities 
of  the  MGSLRT  and  the  MWSLRT.  The  reason  is  that  the  statistics  Adn  are  not  likelihood  ratios 
anymore  so  that  the  change-of-measure  argument  (Wald’s  likelihood  ration  identity)  cannot  be 
applied.  Some  asymptotic  approximations  still  can  be  obtained  in  the  iid  case  for  (-dimensional 
exponential  families  using  large  and  moderate  deviations: 

sup  P g(d  =  j)  =  - — b  -  — - 1-  0(1)  as  min  A^  — >  oo.  (13) 

H,  H,  Aji  ij 
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(cf.  Chan  and  Lai  [1]).  In  the  general  non-iid  case  this  is  still  an  open  problem. 

As  we  mentioned  above,  in  this  respect  the  MASLRT  has  a  big  advantage  over  the  MGSLRT 
to  the  expense  of  some  loss  of  performance  due  to  one-stage  delayed  estimators.  Write  «*;(*)  = 
P e(d*  =  j)  UjflcQ,  }  for  the  error  probabilities  of  the  MASLRT.  We  now  show  that  supfl60.  a£-(0)  ^ 
1  fAij,  i  j,  so  that  =  1/a ij  implies  5*  G  C(| \aij\ |). 

Theorem  1.  Let  af{6)  =  Pe(d*  =  j)t{9eQi}  and  /3*(6)  =  P 9(d*  f  i)l{ee0i},  i  =  0, 1, . . . ,  N  be 
the  error  probabilities  of  the  MASLRT  6*  =  ( d*,T *).  The  following  inequalities  hold: 

(i)  sup0e0.  af(6)  ^  l/Ai:jfori,j  =  0, 1, . . . ,  N,  i  ±  j  ; 

(ii)  sup0e0.  p*  ( 6 )  ^  Afl  for  i  =  0, 1, . . . ,  W 

Proof  Since  {d*  =  j}  =  {T*  —  T* }  implies  {T*  <  cx)},  we  have 


a*ij{0)  —  Eel{d*=j}  ^  Eell 


{t;<  00} 


=  Ef. 


(6) /MO) 


for  all  0  G  0,. 


By  the  definition  of  T*,  AL*  ^  eaij  and  clearly  Ar~  (6)  f  AL*  for  all  6  G  0 , .  Therefore,  for 

J  j  J  j 

all  0  G  0j, 


««(*)  ^  E0  fl{r*<oo}  At;(6»)/At;(6») 


<  e  E<: 


1{T*  <00)^2 


?(0) 


=  e 


where  the  last  equality  follows  from  the  Wald  likelihood  ratio  identity.  This  proves  (i). 
Part  (ii)  follows  immediately  from  the  fact  that 


and  the  proof  is  complete.  □ 

Therefore,  we  have  the  following  important  implications: 

Aij  =  l/aij^6*  gC(|K||);  (14) 

Aij  =  Ai  =  N/ Pi  =>  5*  G  C (/3).  (15) 

We  note  that  generally  there  are  no  such  inequalities  for  the  MGSLRT  and  the  MWSLRT. 
2.1.3.  Near  Optimality 

The  developed  asymptotic  hypothesis  testing  theory  is  based  on  the  SLLN  and  rates  of  convergence 
in  the  strong  law  for  the  LLR  processes,  specifically  by  strengthening  the  strong  law  into  the  r- 
quick  version. 

Definition  1.  For  r  >  0,  the  random  variable  fn  is  said  to  converge  P -r-quickly  to  a  constant  C  if 
E Lre  <  00  for  all  e  >  0,  where  Le  =  sup  {n  :  |£n  —  C\  >  e}  (sup  0  =  0). 
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Note  that  P (Le  <  oo )  =  1  for  all  £  >  0  is  equivalent  to  the  P-a.s.  convergence  of  to  C. 
Write  A n(0,  6)  =  log  =  Y2=  i  loS  for  the  log-likelihood  ratio  (LLR)  process. 

~Q  P()\^k  1-^-1  ) 

Assume  that  there  exist  positive  and  finite  numbers  1(6 ,  6)  such  that 

-A JO, 0)  P---f--q---y>  1(0, 0)  for  all  6,6  E  Q,  6  f  6.  (16) 

77,  n— >oo 

In  addition,  we  certainly  need  some  conditions  on  the  behavior  of  the  estimate  6n  for  large  n, 
which  should  converge  to  the  true  value  6  in  a  proper  way.  To  this  end,  we  require  the  following 
condition  on  the  adaptive  LLR  process: 

-  log  AJ0)  ■--■r-^-qu-i-c-->  1(6. 0)  for  all  6,6  6  0,  0^0,  (17) 

Tl  n— >•  oo 

so  that  the  normalized  by  n  LLR  tuned  to  the  true  parameter  value  and  its  adaptive  version  converge 
to  the  same  constants.  In  certain  cases,  but  not  always,  conditions  (16)  and  (17)  imply  the  following 
conditions 

-log  A*  p»-r-q"iddy>  /.(fl)  for  all  0  e  ©  \  0*,  i  =  0, 1, . ,N,  (18) 

77,  n— >oo 

where  7(0)  =  infgge.  1(6,6),  the  minimal  “distance”  from  6  to  the  set  0,  is  assumed  to  be 
positive  for  all  i.  Let 


7(0)  =  min  [I j (6) /c-ji\  for  6  E  ©,;,  J(0) 

O^j^N 


max  Ji(6)  for  6  E  Iin, 

O^N 


(19) 


and 


J*(6)=  min  [lj(0)/cj]  for  0  E  0,,  J*(6) 

O^j^N 


max  min  17,(0) /ml 
j¥=i 


max  J* (6)  for  6  E  I, 


0<i<N 


(20) 


where =  limamax^0  I  \ogaVJ\/\  log  ^max|?  ^max  maxjj  otij,  Ci  linijamax_5.o  |  log  /7|/|  log  /3max|, 

fimax  lliaXj  3j. 

The  following  theorem  establishes  uniform  asymptotic  optimality  of  the  MASLRT  in  the  gen¬ 
eral  non-iid  case  with  respect  to  moments  of  the  stopping  time  distribution.  The  proof  is  based 
on  the  technique  developed  by  Tartakovsky  [5]  for  multiple  simple  hypotheses.  It  is  very  lengthy 
and  therefore  omitted.  We  just  mention  that  it  includes  a  two-step  procedure:  first  to  obtain  the 
asymptotic  lower  bounds  for  moments  of  the  stopping  time  distribution  inf5ec(||aij||)(C(/3))  E o[T]m, 
6  E  0,,  m  >  0,  i  =  0, 1, . . . ,  N,  and  then  to  show  that  these  lower  bounds  are  attained  for  the 


procedure  of  interest. 


Theorem  2  (Asymptotic  Optimality).  Assume  that  r-quick  convergence  conditions  (16)  and  (18) 
are  satisfied. 

(i)  If  the  thresholds  Aij  are  so  selected  that  sup0ge.  q/?(0)  7  and  log  At]  ~  !og(l/a:,?),  in 
particular  AtJ  =  l/a-y,  then  for  m  7  r  as  amax  — >  0 


inf  E eTm  ~ 

<5e€(||aij||) 


E e[T*]m  ~ 


log  amax|/ Ji(6)]m  for  all  6  E  0,  and  i  —  0, 1, . . . ,  N 
log  «max|/ J(6)]m  for  all  6  E  Iin, 
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where  the  functions  JfO),  J{0)  are  defined  as  in  (19). 

(ii)  If  the  thresholds  Aij  =  /l,  are  so  selected  that  sup0g0!  ft*  (0)  f  ff  and  log  A*  rs_/  log(l  /fa,  in 
particular  Ai  =  N/  ft,  then  for  m  f  r  as  /jrnax  — >  0 


inf  E eTm  ~  Ee[T*]m  ~ 

<5eC(/3)  -  - 


log/3max\/J*(d)]m  for  all  0  G  0.t  and  i  =  0, 1, . . .  ,N 
far  all  0  G  Iin, 


where  the  functions  J*(0),  J*(B)  are  defined  as  in  (20). 

Consequently,  the  MASLRT  minimizes  asymptotically  the  moments  of  the  sample  size  up  to 
order  r  uniformly  in  Q  G  0  in  the  classes  of  tests  C  ( 1 1  or]  \  \ )  and  C  ((3). 

This  theorem  generalizes  previous  results  of  Pavlov  [4]  and  Dragalin  and  Novikov  [2]  restricted 
to  iid  exponential  families,  and  also  provides  alternative  conditions  in  iid  cases  that  can  be  often 
easily  checked. 

Remark  1.  The  assertions  of  Theorem  2  of  course  also  hold  for  the  MGSLRT  and  MWSPRT 
when  the  r-quick  convergence  conditions  (18)  are  satisfied  for  the  GLR  statistics  An  (  0,  )  =  tn  —  tln 
and  the  mixtures  X'n.  However,  we  stress  that  there  are  no  simple  upper  bounds  for  the  error 
probabilities  of  the  MGSLRT  and  the  MWSPRT.  Furthermore,  while  for  iid  exponential  fami¬ 
lies  certain  asymptotic  approximations  for  the  error  probabilities  can  be  obtained  based  on  the 
boundary-crossing  framework  and  large  deviations  (see  Chan  and  Lai  [1],  Lorden  [3]),  for  general 
non-iid  models  no  such  results  exist. 

Remark  2.  The  assertions  of  Theorem  2  remain  true  if  the  normalization  by  n  in  (18)  is  replaced 
with  the  normalization  by  f(n),  where  wit)  is  an  increasing  function,  ip( oo)  =  oo,  in  which  case 
[|  logamax|/Jj(0)]m  in  (21)  should  be  replaced  with  \l/([|  log  amax|/ where  'k  is  inverse 
to  and  similarly  in  (22). 

We  now  consider  two  interesting  examples. 

2.1.4.  Testing  for  the  Gaussian  Mean  with  Unknown  Variance 

Consider  the  Gaussian  example  assuming  that  Xn  ~  Af(p,  a2),  n  —  1,2,...  are  iid  normal  random 
variables  with  unknown  mean  p  and  unknown  variance  a2  and  the  hypotheses  are  IIo  :  //  A 
p0,  a2  >  0  and  Hi  :  p  ^  pi,  a2  >  0,  where  pi,  po  are  given  numbers,  p\  >  p0.  The  variance  o2  is 
a  nuisance  parameter. 

In  the  following  we  consider  a  specific  case  H0  :  p  =  0  and  H\  :  p  f  p\  (p\  >  0).  This 
problem  is  of  special  interest  in  certain  applications.  For  example,  when  detecting  targets  in 
noise/clutter  the  observations  have  the  form  Xn  =  p  +  Vn  if  there  is  a  target  and  Xn  =  Vn  if 
there  is  no  target.  The  value  of  p,  p  >  0  characterizes  the  intensity  of  the  signal  from  the  target; 
Vn  is  sensor  noise  or  clutter  plus  noise.  Assuming  that  {Vn}n^i  is  zero-mean  white  Gaussian  noise 
with  unknown  variance  a2,  we  arrive  at  this  problem.  In  radar  applications,  p  usually  represents 
the  result  of  the  preprocessing  by  attenuation  and  matched  filtering  of  the  modulated  pulses  and, 
also,  is  not  known.  The  value  of  p\  >  0  is  a  prespecified  limit  or  cut-off  intensity  of  the  target.  In 
this  interpretation  the  value  of  q  =  p/a  represents  an  unknown  signal-to-noise  ratio  and  q\  =  p\/o 
is  a  given  cut-off  signal-to-noise  ratio  level.  Thus,  we  are  dealing  with  the  two-hypotheses  prob¬ 
lem  (N  =  1)  for  the  two-dimensional  exponential  model  with  the  parameter  6  =  (p,  a2)  and 
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parameter  space  0  =  [0,  oo)  x  (0,  oo).  Also,  0O  =  {0}  x  (0,  oo),  @i  =  |/ti,oo)  x  (0,  oo),  and 
Iin  =  (0,/ii)  x  (0,  oo). 

We  now  show  that  all  the  conditions  of  Theorem  2  are  satisfied  when  {6n}  is  a  sequence  of 
MLEs,  which  implies  the  uniform  asymptotic  optimality  of  the  2-ASPRT  with  a*  =  iogfl/a-.,.), 
i  =  0, 1^ 

Let  9  =  ( Jl ,  a2),  where  Jl  and  a  are  arbitrary  numbers,  Jl  ^  0,  a  >  0.  Then  the  LLR  A n(0,  9)  = 
L{9)  ~  4(0)  =  ELi  log \pe{Xk)/pg(Xk)\  is  given  by 


K{0,0)  =  ^log  (^2  )  +  2?2a2 


9  n 

crz  -  az 


Ex 


k= 1 


— 9  ~  O  71  9~9  ~-9  9 

fi<7~  —  fia  /i  a  —  fi  a 

>  yAfc - o - n. 


(23) 


?V2 


fc=i 


2(j2(T2 


Using  (23),  it  is  not  difficult  to  show  that 


1(0,9)  =  ^  {[(/^  —  /^)2  +  cp2]/cr2  -U  log(cx2/cr2)  —  !}  -  (24) 

The  minimum 

min/(0,  9)  =  ]-  log  [l  +  (fi  -  Jl)2/v2} 

(T>0  z 

is  achieved  at  the  point  a*  =  [cr2  +  (ji  —  Jl)2}1/2  and  h{6)  =  min^Ml  min^o  1(6,  6)  and  Jo(0)  = 
mingefo}  min5F>0  /(0,  6)  equal 


T  (  \  ^St1  +  (?i  -  Q)]  for  0  <  q  <  qi 
1m)  ~  \  n  f  ^  ’ 

[0  for  q  ^  qi  (25) 

Iq(q)  =  ^log(l  +  q2)  forg^O, 

where  q  —  n/ a  and  q1  =  n\/a.  Thus,  as  expected,  the  initial  two-dimensional  hypothesis  testing 
problem  is  reduced  to  the  equivalent  single -parameter  testing  problem  H0  :  q  =  0  against  //,  :  q  E 
qi  with  the  parameter  space  Q  =  [0,  oo)  and  subsets  Q 0  =  {0},  Qi  =  [qi,  oo),  Iin  =  (0,  q\). 

Clearly,  I0(q)  >  0  for  q  e  Qi+hn  =  (0,  oo)  and  h(q)  >  0  for  q  e  Qo+hn  =  [0,  qi),  and  hence, 
min[/0(g),  h(q)]  >  0  for  q  e  Iin  =  (0,gi).  Also,  it  is  easily  verified  that  infgeQ  max[/0(g),  h{q)c]  > 
0  for  any  0  <  c  <  oo.  In  fact,  the  maximum  is  attained  at  the  point  q*  e  (0,  <{\ )  for  which 
J0 (q*)  =  I\(q*)c,  and  it  is  a  solution  of  the  equation 

(l  +  q2Y/c  =  l  +  (qi-q)2.  (26) 

In  particular,  q*  =  qi/2  and  inf^gQ  max[/0(g),  h{q)c]  =  log(l  +  qj/ 4)  for  c  =  1.  Therefore,  the 
conditions  related  to  the  minimal  Kullback-Leibler  (  K-L)  distances  for  the  corresponding  sets 
hold,  and  it  remains  to  deal  with  convergence  of  the  LLR  and  associated  statistics. 

Not  surprisingly  we  choose  9n  =  (Jin,  a2)  as  the  maximum  likelihood  estimator, 

(An,  £n)  =  arg  sup  A n(/x,  a2,  Jl,  a2), 

/igO, 
a2>  0 
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which  is  of  course  a  combination  of  the  positive  part  of  the  sample  mean  and  sample  variance,  i.e., 


An  =  max{0,  Xn},  =  n  1  -  An)2, 

k= 1 

where  Xn  =  n_1  Y2k=i  's  the  sample  mean. 

First,  since  the  LLR  {Xn(0,0)}n^i  given  by  (23)  is  a  random  walk  with  drift  1(0,6)  and 
Ee|Ai(0,  6)\r  <  oo  for  all  positive  r,  it  follows  that 

n~lXn(0,  0)  -Pe-:-^c-k-^->  1(0 ,  6)  for  all  r  >  0. 

71— >00 


Hence,  the  conditions  (16)  are  satisfied  with  1(6 , 6)  equal  the  K-L  numbers  given  by  (24)  and 

0  =  (/fi  o-^- 
Write 

An  =  log  A^,  Xn(0)  =  log  A„(0). 

Note  that  A),  =  An  —  A ln,  i  =  0, 1.  Since  X1,  X2, ...  are  iid  and  E0|  AA  |r  <  oo  for  all  r  >  0,  the 
following  r-quick  convergence  conditions  hold  as  n  — >  oo  under  P0: 

jin  — ^  /b  An  — ^  A ^ i  ^  ^ i  ^n,0  — ^  A  /A,  V/i  ^  0,  O"2  >  0, 

{/i  if  n  ^  // 1  2  J  cr2  if  /!  ^  /i! 

•fn  /  ’  an,l  )  2  ,  /  \2  n  / 

// 1  if  0  ^  fi  <  Hi  I  a  +  (/i  —  /ii)  if  0  ^  fi  <  fi\. 


Using  these  relations,  it  can  be  verified  that  P0-r-quickly  as  n  — >■  oo 

n~l  A*  — >•  (/i2  +  a2  —  log  a2  —  l)  /2,  V/i  ^  0,a2  >  0; 
n-1A°  — >  [/i2  +  a2  —  log(//2  +  a2)  —  l]  /2,  V/i  ^  0,  tr2  >  0; 


n 


,-i\i 


K 


(/i2  +  tr2  -  logtr2  -  1)  /2 

{/i2  +  o-2  +  log[o"2  +  (/i  -  /ii)2]  -  1}  /2 


if  /i  ^  /ii,  o-2  >  0 
if  0  ^  n  <  /ii,  a2  >  0. 


Combining  these  formulas  yields 


n-1  A*  ------4  A(0),  0  e  0  \  0*,  i  =  0, 1  for  all  r  >  0, 

71— >-00 


(27) 


where  h(0)  =  min^  min j>0 1(0,0)  =  h(q )  and  Jo(0)  =  min?e{0}  mina?>0  1(0,  0)  =  J0(g) 
are  given  by 

Ji(g)  =  1/2  log[l  +  (qi  -  q)2}  for  0  ^q  <  qx\  I0(q )  =  1/2  log(l  +  q2)  for  q  >  0, 


q  =  li/cr,  qi  =  iii/a  (cf.  (25)). 

Therefore,  the  conditions  (18)  are  satisfied  with  /,($)  =  h(q)-  By  Theorem  2,  the  2-ASPRT 
is  asymptotically  uniformly  optimal  in  the  sense  of  minimizing  all  the  positive  moments  of  the 
stopping  time  distribution:  for  all  r  ^  1  as  amax  — >•  0 


inf  E0Tr  ~  E0[T*lr  ~ 

5eC(aom) 


{2 1  log  ai\/  log[l  +  (qi  —  q)2)}r 
{2 1  log a0|/log[l  +  g2]}r 


if  0  ^  q  ^  q* 
if  q  ^  q*. 
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In  addition, 

inf  sup  E eTr  ~  E e*[T*]r  ~  (2|  loga0|/log[l  +  (<f  )2]}r, 

6eC(ao,ai)  0e© 

where  q*  G  (0,  qf)  is  the  solution  of  the  equation  (26). 

These  asymptotics  are  also  true  for  the  GSLRT  and  the  WSPRT  if  the  thresholds  are  selected 
so  that  the  logarithms  of  the  error  probabilities  of  these  tests  are  asymptotic  to  log  al. 

2.1.5.  Testing  for  a  Nonhomogeneous  AR  Sequence 


Let 


Xn  —  0  ■  Sn  +  £n)  n  —  1,2, ... , 


where  Sn  is  a  deterministic  function  and  {£n}n^i  is  a  stable  first-order  AR  Gaussian  sequence 
given  by  the  recursion 

£>n  7  £n— 1  +  Cnj  n  ^  1, 

where  Ci,  C2,  •  •  •  are  iid  Gaussian  variables,  (k  ~  A/”(0,  a2),  and  |^y |  <  1.  For  the  sake  of  concrete¬ 
ness  we  set  £0  =  0  and  S0  =  0,  while  all  the  results  are  true  for  arbitrary  deterministic  or  random 
initial  conditions.  The  hypotheses  are  H0  :  9  ^  90  and  If  :  9  f  9\,  where  90  <  0\  are  given 
numbers.  That  is,  0  =  (— 00,  00),  @0  =  (—00,  0O],  0i  =  [Q\,  00),  Iin  =  (90,  Of.  In  this  case,  the 
LLR  can  be  written  in  the  form 


A  n(0,9) 


9-9 


n 

Y.SkXk 

k= 1 


92-92 

2a2 


n 


E-^ 


where  Xk  =  Xk  —  ^Xk_y,  Sk  =  Sk  —  7,S'/,_|  (X0  =  ,S0  =  0).  Direct  computation  shows  that 

{9  -  0)2 


E0[A  n{9,9)\  = 


2a2 


E3- 


k= 1 


Suppose  that 


lim  n  1  ^  Si  =  S2, 

rt. — Vnn  *  ^ 


(28) 


k= 1 


where  S2  is  a  positive  and  finite  number.  Then,  for  all  9,0  e  {—00,  00)  (9  f  9), 

_  _ 9)2s2 

n~l\n{9,  9)  — y - — — -  P e  —  r  —  quickly  for  all  r  >  0. 

2al 


(29) 


Indeed,  under  Pg,  the  whitened  observations  Xn  can  be  written  as  Xn  =  9Sn  +  Qn  and  the  LLR  as 


(7 


k=  1 


where  Vn  =  Skfk  is  a  weighted  sum  of  iid  normal  random  variables.  Since  EVn  =  0 

and,  by  (28),  for  large  n  EV2  ~  S2n,  it  is  obvious  that  there  exists  a  number  <5  <  1  such  that 
P(|K|  >  £n)  f  0(8n),  which  yields 


OO 

yy  nr_1P(|14|  >  en)  <  00  for  some  £  >  0  and  all  r  >  0.  (30) 

71=1 
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In  other  words,  Vn/n  converges  to  0  r-quickly,  which  implies  (29).  Hence,  the  conditions  (16) 
hold  with  1(9,  9)  =  (9  —  9)2S2/2cr2,  and  it  remains  to  check  the  conditions  (18),  where 


h(0)  =  inf  1(9,9) 

9^9i 

Io(9)=  inf  1(9,9) 

9^9o 


(< 9 1  -  9)2S2 

2^ 

(9  -  90)2S 2 

2^ 


for  9  <  9 1 , 
for  9  >  90. 


(31) 


As  the  estimate  9n,  we  of  course  use  the  MLE  Ylk=i  SkXk/Ylk= i  Moreover  let  0n,  |  = 
max(0!,  9n)  and  9n  i]  =  min(#0,  9n).  Then  the  statistics  X'n  can  be  written  as 

-t  n  1  n 

K=-> £(4-.  - Kt&x* - ^3 E<C. - X)sl  i-o,i. 

k=  1  k= 1 

Using  an  argument  similar  to  that  has  led  to  (30)  with  a  minor  generalization,  we  conclude  that, 
r-quickly  under  Pe, 

9 n  9,  0n, i  — >  max(9\,  9),  0n, o  — >  max(#o,$)>  92n  — >  92 , 

n 

0^, i  max(6>i,  02),  O2n0  ->•  max(6>o,  92),  n~l  ^  SkXk  ->•  6»2S'2, 

fc=i 

which  after  some  manipulations  yield 

Li(0),  9  e  0  \  0,;,  i  =  0, 1  for  all  r  >  0, 

n— >•  oo 

where  the  Ii(9)’s  are  given  by  (31). 

Thus,  by  the  Theorem  2,  the  2-ASPRT  is  asymptotically  optimal,  minimizing  all  the  positive 
moments  of  the  sample  size:  for  all  r  ^  1  as  amax  — >  0, 

inf  E„r  ~E4TT~|{2|l0S“ll/[(5l“,)2-21r  if?<9' 

<5eC(a0,ai)  |  (2|  log  a0 1  /  [(g  -  go)2#2]  y  if  q>q*, 

where  q  =  9  jo,  qi  —  9 \/a,  and  q*  is  a  solution  of  the  equation 

|  loga0|/[(g  -  g0)2  =  I  logc*i|/(gi  -  g)2. 

In  particular,  if  Sn  —  1,  then  S2  =  (1  —  ry)2. 
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